Distributed vector architectures

Integrating processors and main memory is a promising approach to increase system performance. Such integration provides very high memory bandwidth that can be exploited efficiently by vector operations. However, traditional vector applications would easily overflow the limited memory of a single integrated node. To accommodate such workloads, we propose the DIstributed Vector Architecture (DIVA), that uses multiple vector-capable processor/memory nodes in a distributed shared-memory configuration, while maintaining the simple vector programming model. The advantages of our approach are twofold: (i) we dynamically parallelize the execution of vector instructions across the nodes, (ii) we reduce external traffic, by mapping vector computation—rather than data—across the nodes. We propose run-time mechanisms to assign elements of the architectural vector registers on nodes, using the data layout across the nodes as a blueprint. We describe DIVA implementations with a traditional request-response memory model and a data-push model. Using traces of vector supercomputer programs, we demonstrate that DIVA generates considerably less external traffic compared to single or multiple-node alternatives that are based solely on caching or paging. With timing simulations we show that a DIVA system with 2 to 8 nodes is up to three times faster than a single node using its local memory as a large cache and can even outperform a hypothetical system where the application fits in local memory.